A First-Order Approach to Accelerated Value Iteration

نویسندگان

چکیده

Markov decision processes (MDPs) are used to model stochastic systems in many applications, but computing good policies becomes hard when the effective horizon become very large. In “A First-Order Approach Accelerated Value Iteration,” Goyal and Grand-Clément present a connection between value iteration (VI) algorithms gradient descent methods from convex optimization use acceleration momentum design faster algorithms, with convergence guarantees for computation of function fixed policy reversible MDP instances. The authors provide lower bound on properties any first-order algorithm solving MDPs, where no can converge than VI. Finally, introduce safe accelerated (S-AVI), which alternates updates updates. S-AVI is worst-case optimal retains theoretical VI while exhibiting strong empirical performances providing significant speedups compared classical approaches large test bed

برای دانلود باید عضویت طلایی داشته باشید

برای دانلود متن کامل این مقاله و بیش از 32 میلیون مقاله دیگر ابتدا ثبت نام کنید

اگر عضو سایت هستید لطفا وارد حساب کاربری خود شوید

منابع مشابه

Accelerated Extra-Gradient Descent: A Novel Accelerated First-Order Method

We provide a novel accelerated first-order method that achieves the asymptotically optimal con-vergence rate for smooth functions in the first-order oracle model. To this day, Nesterov’s AcceleratedGradient Descent (agd) and variations thereof were the only methods achieving acceleration in thisstandard blackbox model. In contrast, our algorithm is significantly different from a...

متن کامل

A numerical approach to solve eighth order boundary value problems by Haar wavelet collocation method

In this paper a robust and accurate algorithm based on Haar wavelet collocation method (HWCM) is proposed for solving eighth order boundary value problems. We used the Haar direct method for calculating multiple integrals of Haar functions. To illustrate the efficiency and accuracy of the concerned method, few examples are considered which arise in the mathematical modeling of fluid dynamics an...

متن کامل

A Lyapunov approach to Accelerated First Order Optimization In Continuous and Discrete Time

A Lyapunov Approach to Accelerated First-Order Optimization In Continuous and Discrete Time

متن کامل

Accelerated First-order Methods for Hyperbolic Programming

A framework is developed for applying accelerated methods to general hyperbolic programming, including linear, second-order cone, and semidefinite programming as special cases. The approach replaces a hyperbolic program with a convex optimization problem whose smooth objective function is explicit, and for which the only constraints are linear equations (one more linear equation than for the or...

متن کامل

ذخیره در منابع من


  با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید

ژورنال

عنوان ژورنال: Operations Research

سال: 2023

ISSN: ['1526-5463', '0030-364X']

DOI: https://doi.org/10.1287/opre.2022.2269